Skip to content

Fix MS Teams Support#1

Open
anx-ag wants to merge 42 commits intomasterfrom
msteams-fix
Open

Fix MS Teams Support#1
anx-ag wants to merge 42 commits intomasterfrom
msteams-fix

Conversation

@anx-ag
Copy link
Copy Markdown
Owner

@anx-ag anx-ag commented Mar 12, 2026

Major MSTeams Overhaul

Both directions tested and working:

  • Threading (including proper thread reply mapping across bridge restarts via persistent cache)
  • Emoji support (with cross-platform emoji mapping table, e.g. :flag_at: vs :flag-at:)
  • Code Block support including syntax highlighting (Mattermost->Teams does not support syntax highlighting as far as limited research revealed)
  • Code Snippet relay (Teams code snippet attachments are converted to markdown fenced code blocks)
  • Message editing
  • Message deletion (soft-delete via Graph API)
  • Formatting (bold, italic, strike-through, headings, lists, blockquotes, etc.)
  • Online GIFs (Giphy, etc.)
  • Images (even multiple in the same post), no GIFs via hostedContents (GIFs sent as URL)
  • Works with posts and threads layout in Teams

Teams -> Mattermost:

  • Files (including warning message posted in Teams when the file size exceeds the MediaDownloadSize configuration limit)
  • GIFs
  • Inline images via hostedContents API download

Mattermost -> Teams:

  • Message Priorities (Urgent & Important) -- forwarded as emoji prefix indicators
  • Images via hostedContents API (PNG/JPG embedded directly, no external server needed)
  • Other files via MediaServer upload or source URL pass-through

Message Replay / Persistent Cache (new):

  • New configuration parameters: MessageCacheFile and MessageCacheDuration (default: 7 days)
  • The bridge persists thread/post ID mappings and delta tokens to a local JSON file
  • After startup, the bridge checks for new messages since the last cached delta token and replays them with a [Replay YYYY-MM-DD hh:mm TZ] prefix
  • Deduplication prevents replaying already-bridged messages
  • Time-based cache pruning keeps the cache file manageable

Graceful Shutdown (new):

  • Signal handling (SIGINT/SIGTERM) triggers orderly shutdown
  • All persistent caches are flushed to disk before exit (synchronous Stop())

Delta Query Polling (new):

  • Replaced the old polling loop with Microsoft Graph /messages/delta API endpoint
  • Faster, more reliable message detection with less API overhead
  • Delta-guided reply polling for thread replies

Changes:

  • UI/UX fixes: [thread] and RemoteNick prefixes no longer show up redundantly when threading is working
  • Automatic background OAuth token refresh to support editing/deleting messages after token expiry
  • {DISPLAYNAME} placeholder for RemoteNickFormat (uses "Firstname Lastname" from Mattermost)
  • Historical message ID mapping embedded in message HTML for cross-restart thread resolution
  • Security hardening: HTML-escape filenames, URL-encode media server paths, validate code snippet URLs, consistent channel ID decoding

Features:

  • Test Mode: typing @matterbridge test on either side starts an automated test sequence covering formatting, emojis, threads, edits, deletions, images, priorities, etc.
  • Emoji Mapping Table: configurable regex-based emoji shortcode mapping between platforms (e.g. :flag-at: <-> :flag_at:)
  • Persistent Message Cache: MessageCacheFile stores post/thread ID mappings in a local JSON file, surviving bridge restarts. MessageCacheDuration (default 7d) controls cache entry expiry.
  • Hybrid Webhook + Token Mode (Mattermost): supports both webhook and token-based authentication simultaneously

Known Issues:

  • Code Blocks do not use Syntax Highlighting in Teams (Teams limitation)
  • @matterbridge test postings from Teams to Mattermost may arrive out of order because image upload takes longer than the sleep intervals between posts

Todo:

  • File Support (Mattermost -> Teams) for non-image files without MediaServer
  • Acknowledgements (enterprise license needed)
  • Scheduled Messages (enterprise license needed)
  • MediaServer support for file and GIF sharing from Mattermost to Teams
  • Teams messages received from Mattermost should use the avatar configured with IconURL
  • Ping usernames across the bridge

anx-ag and others added 30 commits March 12, 2026 05:38
Webhooks in Mattermost allow for overriding the username and icon in root messages, bot tokens support threading. We need to use both in hybrid mode for the best user experience.
This patch fixes multiple issues and adds new features.
* Support for message editing (bi-directional)
* Support for message deletion (bi-directional)
* Initial support for image/file attachments (not working ATM)
* Since we support threading, the "[thread]" prefix is not posted anymore
* Message formatting is fixed so that the nick is posted in a separate bold line
* Initial support for some markdown (code, quote, etc.) - some of it works bidirectional at the moment, some only in one direction (markdown2teams converter and vice versa)
* ignores messages before matterbridge starttime
* keeps track of sent messages during runtime
Posts a sequence of test messages (root, thread replies, code block,
quote, emojis, formatting) with edit and delete steps when a user
types "@matterbridge test" in either Mattermost or Teams. The trigger
message is intercepted and not relayed; the test messages bypass echo
prevention so they flow through the normal relay pipeline.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Rewrite mdToTeamsHTML() to use gomarkdown for full markdown→HTML
  conversion (bold, italic, strikethrough, headings, links, quotes,
  code fences, line breaks). Previously only code fences were converted.
- Add HTML-aware RemoteNickFormat expansion: {NICK} renders as <b>bold</b>,
  \n as <br>. Gateway now passes original nick via msg.Extra["nick"].
- Add extensible emoji mapping with regex support (bridge/msteams/emoji.go).
  Converts Mattermost :flag-xx: to standard :flag_xx: format.
- Fix strikethrough Teams→MM: pre-process <s>/<del>/<strike> tags to
  ~~text~~ in convertToMD() before godown processing.
- All messages to Teams now use HTML content type consistently.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Use :flag-at: (hyphen) in Mattermost test instead of :flag_at: (underscore)
- Convert gomarkdown's <del> to <s> in mdToTeamsHTML for Teams strikethrough
- Add debug logging for nick resolution in Send() to troubleshoot RemoteNickFormat

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add ordered and unordered list test messages to both Mattermost and
  MSTeams test sequences (@matterbridge test)
- Implement sendImageHostedContent() using Graph API hostedContents to
  embed images directly as base64 in Teams messages (no external media
  server required)
- Update sendFileAsMessage() to prefer hostedContents for images with
  binary data, falling back to URL-based embedding

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…rid upload

- Strip <img> tags referencing hostedContents URLs in convertToMD() to
  prevent broken markdown images (these are Teams-internal auth-required URLs)
- In hybrid mode, upload files via API before sending text via webhook,
  fixing the issue where file uploads were skipped for top-level messages
- Use CreatePost with webhook-like props (override_username, override_icon_url)
  in handleUploadFile so file messages show the bridged user's identity
- Add bold formatting for PrefixMessagesWithNick in handleUploadFile
- TrimSpace on username to remove trailing newlines from RemoteNickFormat

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Track message IDs from sendImageHostedContent and sendFileAsMessage in
  sentIDs to prevent echo (file messages being relayed back to Mattermost)
- Combine text + image into a single Teams message via captionHTML parameter
  instead of posting them as two separate messages
- Only use hostedContents for JPG/PNG (the only types Microsoft supports);
  skip unsupported types like GIF/WEBP/BMP silently with a log warning
  instead of posting error messages to Teams
- Return (string, error) from sendFileAsMessage and sendImageHostedContent
  for proper ID tracking

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… notifications

- Skip handleAttachments on msg_update/delete events to prevent Teams
  auto-modifications from causing duplicate file downloads (Bug A)
- Return first file message ID from Send() so gateway can cache it for
  thread-reply ParentID mapping (Bug B)
- Add updatedIDs (30s window) alongside sentIDs on all self-posted messages
  to suppress Teams auto-modification echoes as msg_update (Bug C)
- Post a visible notification in Teams channel when a file type is not
  supported by hostedContents (instead of silent drop), with sentIDs/
  updatedIDs protection to prevent relay (Bug D)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ications

- MM bridge: return postID from handleUploadFile so gateway can cache
  it for thread-reply mapping (fixes msg-parent-not-found for file-only
  thread openers from Teams)
- MM bridge: bundle all files into a single post instead of one post
  per file (fixes 3 images → 3 messages from Teams→MM)
- Teams bridge: refactor sendImageHostedContent to accept []FileInfo,
  sending all supported images in one message with multiple
  hostedContents entries (fixes 3 images → 3 messages from MM→Teams)
- Teams bridge: classify files in Send() — supported images go through
  hostedContents batch, others through sendFileAsMessage individually
- Teams bridge: send unsupported file notifications via b.Remote to
  route back to source side (MM) instead of posting to Teams channel

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…tification

The synchronous b.Remote <- msg in sendFileAsMessage blocked forever because
b.Remote is an unbuffered channel read by handleReceive(), which is the same
goroutine that called Send() -> sendFileAsMessage(). Wrapping in a goroutine
lets Send() return immediately so handleReceive() can read the notification
on the next loop iteration.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…erbridge

Return a fake ID from sendFileAsMessage so the gateway caches a BrMsgID entry
for the original source message. The notification references this fake ID as
ParentID, which the gateway resolves back to the original Mattermost post ID
via FindCanonicalMsgID downstream search + getDestMsgID protocol-strip fallback.
This makes the notification appear as a threaded reply to the user's message
instead of a new root message.

Also changes the notification username from "system" to "matterbridge".

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…sage cache

- Translate remaining German text to English in test sequences and notifications
- Add image test steps (PNG, GIF, multi-image) to both MM and Teams test sequences
- Fix isSupportedHostedContentType to include image/gif (supported by MS Graph API)
- Forward MM message priority (important/urgent) to Teams with emoji prefix
- Add persistent JSON-backed message ID cache (MessageCacheFile config option)
  with LRU fallback, write-through, and background flush
- Embed source message IDs in relayed messages (hidden HTML span for Teams,
  matterbridge_srcid prop for MM) for historical cache reconstruction
- Scan recent messages on startup to populate persistent cache from markers
- Add demo.png and demo.gif test assets

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Revert image/gif from isSupportedHostedContentType (Teams API rejects it)
- Add handleHostedContents() to download inline images from Teams messages
  via Graph API hostedContents/$value endpoint for Teams→MM relay
- Replace GIF test step with manual check instruction (Teams uses SharePoint)
- Add priority test steps (important + urgent) to MM test sequence
- Fix MessageCacheFile to use per-bridge config with [general] fallback

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ority tests

- Rework MessageCacheFile from one-per-gateway to per-bridge caches
  with dedup for shared paths and helper methods
- Add from_webhook/override_username/override_icon_url to all API-path
  CreatePost calls so thread replies show bridged user identity
- Remove redundant bold username prefix from handleUploadFile and
  text CreatePost path
- Fix priority test steps: create post first, then SetPostPriority

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…x priority tests, add manual teams file transfer test
…t, autolink fix, priority test fix

- Teams bridge: enforce MediaDownloadSize via HTTP HEAD pre-check + LimitReader fallback,
  notify sender when file exceeds limit (handleDownloadFile, handleHostedContents)
- New DownloadFileWithSizeCheck helper with ErrFileTooLarge error type
- Message replay: fetch and relay missed messages on bridge restart using
  configurable ReplayWindow (per-bridge with [general] fallback), LastSeen
  tracking in PersistentMsgCache, dedup via persistent cache, thread preservation
- Priority test: send priority posts as root posts (Mattermost requires this),
  reorder steps so delete runs before priority, "Test finished" stays last
- Autolink: add parser.Autolink to mdToTeamsHTML so plain URLs from Mattermost
  become clickable <a> tags in Teams

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…erby from replay API

- notifyFileTooLarge: use Graph API reply directly instead of b.Remote,
  so the warning appears in the Teams channel where the file was uploaded
  (b.Remote would route it to Mattermost instead)
- Teams replay: remove $orderby=lastModifiedDateTime+desc from Graph API
  URL — not supported by the messages endpoint, client-side sort suffices

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…blocking

replayMissedMessages was called synchronously before the poll goroutine,
causing the poll loop to never start if the Graph API call hung or
b.Remote send blocked. Now runs inside the goroutine like Mattermost does.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…stent log levels, test icons

- replayMissedMessages now fetches thread replies via getReplies() with
  correct ParentID and composite key (msgID/replyID) matching poll loop
- Add @odata.nextLink pagination (max 5 pages) to guarantee ReplayWindow
  coverage beyond 50 messages
- Skip empty replay messages (no text + no files after attachment processing)
- Change handleAttachments log level from ERROR to WARN for download failures
  (consistent with handleDownloadFile and GIF unsupported warnings)
- Add from_webhook + override_icon_url to Mattermost test messages so they
  use the configured IconURL instead of the default robot icon

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…y timestamps

- Add MarkMessageBridged callback so bridges can persist message IDs
  directly in the cache without routing through the gateway
- notifyFileTooLarge now marks both the original message and its warning
  reply in the persistent cache, preventing re-download and re-relay
  of already-handled messages after restart
- Add timezone (MST format) to replay timestamps for clarity, e.g.
  [Replay 2026-03-13 10:08 UTC] instead of [Replay 2026-03-13 10:08]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…chments

Teams "reply with quote" creates attachments with ContentType=messageReference
that have no Name or ContentURL fields. Added nil checks before dereferencing
these pointer fields to prevent SIGSEGV panic.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove the ReplayWindow config option entirely. Replay now only happens
when the persistent cache has a lastSeen timestamp for the channel
(i.e. the bridge has run at least once before). On first start, no
replay occurs — the cache initializes through normal message bridging.

This prevents the undesired behavior where enabling MessageCacheFile
caused massive replay of historical messages on first start.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace the O(n) getMessages+getReplies polling approach with the
Graph API /messages/delta endpoint. This delivers messages AND replies
in a single API call, eliminating the 12+ second poll delays caused by
sequential getReplies calls per thread.

Delta queries also solve the missed-reply-in-old-threads problem: the
endpoint returns ALL changes since the last sync, including replies in
threads whose root message predates the last poll.

Key changes:
- New fetchDelta() helper with pagination and double-unmarshal for
  replyToId extraction (msgraph.ChatMessage lacks this field)
- Unified poll() handles both replay (stored deltaToken) and normal
  polling via the same delta mechanism
- $deltatoken=latest on first start avoids enumerating all historical
  messages (important for channels with 10k+ messages)
- Remove getMessages(), getReplies(), replayMissedMessages() (no callers)
- Add SetDeltaToken/GetDeltaToken to PersistentMsgCache + bridge callbacks
- Mattermost: add event-type allowlist before debug logging to filter
  status_change/typing/hello noise (respects ShowUserTyping config)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add startTime guard to processDelta so messages created before
poll start are silently seeded instead of relayed. On first start
with $deltatoken=latest, the initial deltaLink returns old messages
that should not be forwarded to downstream bridges.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Log each message from fetchDelta with its type (root vs reply-to:parentID)
and per-page summary. Also log processDelta key/parentID to trace message
flow through the pipeline.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
anx-ag and others added 12 commits March 13, 2026 14:53
The /messages/delta endpoint only returns root messages, not thread
replies. However, when a reply is posted, the parent root message
appears in delta with an updated lastModifiedDateTime.

Use this signal to selectively call getReplies() only for threads
that had activity, instead of polling all threads (O(n) → O(1-2)).

Also seed replies for known root messages on startup to avoid
false-positive relaying on the first poll cycle.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace b.mc.EditMessage() with b.mc.Client.PatchPost() in the
Mattermost bridge Send() to preserve override_username and
override_icon_url Props when editing messages. Previously edits
would reset to the bot's default icon and username.

Also increase delay after image posts in the Teams test sequence
from 1s to 3s to prevent "Test finished" arriving before the
multi-image post.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
PostPatch.Props expects *model.StringInterface, not
model.StringInterface.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Extract FirstName + LastName from Mattermost user profiles and pass
them via Extra["displayname"] to destination bridges. The Teams bridge
expands {DISPLAYNAME} in RemoteNickFormat to show the full name
(e.g. "Alexander Griesser") instead of just the username.

Includes a per-bridge displayNameCache to avoid redundant API calls.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add {DISPLAYNAME} placeholder to gateway modifyUsername() so it
  resolves for all bridges, not just Teams HTML formatting
- Add debug logging to getDisplayName() to show FirstName/LastName/
  Nickname values from Mattermost API
- Add debug logging in Teams Send() to show Extra["displayname"]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add signal handling (SIGINT/SIGTERM) in main() so the bridge flushes
persistent caches before exiting. Previously, stopPersistentCaches()
was defined but never called, causing replay entries to be lost on
kill — leading to duplicate replays on restart.

Add MessageCacheDuration config option (default "168h" = 7 days) to
control how long message ID mappings are kept. Entries older than the
configured duration are pruned hourly during the flush loop and on
startup. Metadata keys (__last_seen__, __delta_token__) are never
pruned. Each PersistentMsgEntry now carries a CreatedAt timestamp.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ogging

MarkMessageBridged stored empty PersistentMsgEntry slices, which prune()
immediately deletes (len==0 check). Store a sentinel entry instead so
marker entries survive across restarts.

Add comprehensive debug logging to trace the replay dedup chain:
- IsMessageBridged: log key lookup result and cache count
- persistentCacheAdd: log key, entry count, and target cache (or SKIPPED)
- Router replay dedup: log cache key, account, and hit/miss
- Cache load: show msg vs metadata entry counts + sample keys
- Cache flush: log entry counts when writing to disk

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Stop() was non-blocking: it closed stopCh and returned immediately.
The flushLoop goroutine would pick up the close and call Flush(), but
main() returned first, killing the goroutine before the write completed.
This caused all cache entries added during the run to be lost.

Add doneCh that flushLoop closes via defer when it returns. Stop() now
blocks on <-doneCh, ensuring the final Flush() finishes before main()
exits.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- HTML-escape filenames in all HTML contexts (img alt, anchor href/text,
  bold tags) to prevent XSS via crafted filenames
- Add decodeChannelID() in updateMessage/deleteMessage for consistent
  channel ID handling in Graph API URLs
- URL-encode filename in uploadToMediaServer() to prevent path traversal
- Add domain validation for code snippet URLs (must be graph.microsoft.com)
- Remove dead Workbook/Worksheets exploration code from findFile()
- Replace spew debug dependency with standard %+v formatting
- Remove verbose nick/displayname troubleshooting logs from Send()

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Teams side: skip messages in processReplay() where the author matches
the bridge's own user ID (botID). This prevents test sequence messages
and relayed messages from being replayed back to their source platform.

Mattermost side: skip messages in replayMissedMessages() that have the
matterbridge_srcid prop, which is set on all bridged messages regardless
of bridge instance UUID. This fixes a race condition where the UUID-based
prop check fails across restarts because the UUID is regenerated.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The botID check broke Teams→Mattermost replay because delegated auth
means botID == the authenticated user's ID, skipping ALL their messages.
Replace with data-mb-src marker check which only matches bridge-posted
messages. Also add the marker to test sequence postRoot/postReply.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move htmlType declaration before closures that reference it. In
SetDeltaToken/SetLastSeen, skip update when value is unchanged to
avoid marking cache dirty on every poll cycle.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@anx-ag anx-ag marked this pull request as ready for review March 13, 2026 18:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant